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ABSTRACT 


Conducting evaluations that are both valid and ethical is imperative for the support and sustainability of programs 
that address underserved and vulnerable populations. A key component is to have evaluators who are knowledgeable 
about relevant cultural issues and sensitive to population needs. Hispanics in rural settings are vulnerable for many 
reasons but especially because of language issues, differences in cultural beliefs and socioeconomic barriers. Additional 
factors, such as cultural proclivity to be deferential to authority figures and the possibility of having an undocumented 
alien status also are important issues for Hispanics living and working in rural communities. Because migrant farm 
workers comprise a large part of Hispanics living in rural areas, attention to the sensitive issues surrounding this 
population is paramount during program evaluations. This paper highlights ethical issues pertinent to program 
evaluations involving Hispanics residing in rural settings. 
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BACKGROUND 

Providing health programs for under- 
served populations, such as Hispanics living 
in rural areas, is an emerging public health 
priority. Because Hispanics are considered 
vulnerable to exploitation,' the need to 
conduct valid and ethical evaluations is 
paramount to the support and sustainability 
of programs directed at this population. Pro- 
gram evaluation is “a collection of methods, 
skills and sensitivities necessary to determine 
whether a human service is needed and likely 
to be used, whether the service is sufficiently 
intensive to meet the unmet needs identified, 
whether the service is offered as planned, and 
whether the service actually does help people in 
need at a reasonable cost without unacceptable 


side Accordingly, evaluators are 

responsible for ensuring that ethical issues 
specific to vulnerable populations are as- 
sessed and that the highest ethical standards 
are applied throughout the evaluation pro- 
cess. This paper discusses some of the lead- 
ing ethical challenges that program evalua- 
tors face when carrying out their work with 
Hispanics in rural communities, including 
issues with informed consent, literacy and 
language barriers, cultural beliefs, migrant 
status, alien or undocumented status, and 
advocacy and evaluation roles. 

HISPANICS IN RURAL SETTINGS 

Hispanics account for 25% of the popu- 
lation growth in nonmetropolitan areas.’ 


About half of U.S. Hispanics have settled in 
rural areas that are commonly dominated by 
non-Hispanic whites.’ According to the U.S. 
Department of Agriculture,'* about 34% of 
Hispanics are foreign-born, 53% have a high 
school education, 43% work in agriculture. 
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construction, or manufacturing, and 26% 
live at the poverty level. The largest ethnic 
group to reside in rural areas is comprised 
of Mexicans, most of whom are migrant 
farmworkers.^ Included in this group of 
farmworkers are 1 1.1 million undocumented 
migrants.^ The majority of migrant workers 
arrive from economically depressed areas of 
Mexico in search of job opportunities or a 
higher quality of life.^’*’ Many have little for- 
mal education^ and rate low in both English 
and Spanish literacy.*’’^ These factors limit 
employment opportunities, and therefore, 
result in less desirable or high-risk jobs such 
as working with hazardous chemicals or those 
with increased injury exposure.® 

VULNERABILITY TO EXPLOITATION IN 
PROGRAM EVALUATION 

In general, Hispanics are considered a 
vulnerable population because of language 
barriers, socioeconomic barriers, educa- 
tion differences, documentation status and 
different cultural belief systems.'’’’’'” This 
generalization notwithstanding, Hispan- 
ics residing in rural settings are at an even 
greater disadvantage than urban Hispanics 
because of the increased likelihood of their 
lacking access to care, having undocumented 
alien status and being at enhanced risk for 
occupational exploitation.'’^’’’"’'^ 

ETHICAL CONSIDERATIONS 

Phoenix'® questioned whether current 
human protection programs were sufficient 
to protect the poorly educated, those with 
low social economic status and members of 
ethnic minority groups. The most common 
concerns included: (1) comprehension of 
the informed consent process with ethnic 
minorities with low or no English literacy, 
and (2) payment or remuneration that is not 
coercive. Although these points are valid ones 
to consider in program evaluations involving 
rural communities, there may be many other 
issues related to their vulnerability. 

Whereas the examples that follow focus 
principally on Mexican migrant farmwork- 
ers, the ethical issues apply to most other 
vulnerable populations. However, the cul- 
tural issues strongly embedded with Mexican 


farmworkers and other Hispanic migrants 
make these ethical issues especially relevant 
to these groups. The most notable issues 
involve trust, informed consent, literacy and 
language barriers, cultural-related issues, the 
migrant nature of many rural Hispanics, the 
role conflict of advocacy versus evaluation, 
and other evaluation concerns. 

Trust 

A fundamental value among Hispanics is 
trust {confianza}}*'^^ An evaluator who is not 
indigenous to the culture might encounter 
resistance from program recipients if trust 
is not established prior to the evaluation.’’*'' 
This concern of trust may be explained, in 
part, by documentation status, the task of 
obtaining personal health information (es- 
pecially from someone outside of the fam- 
ily), cultural deviance (i.e., behaviors that 
are against social norms or risky behaviors), 
and poverty.'” 

The use of gatekeepers, people who can 
provide access to a community, is one mech- 
anism for establishing trust.'’’’’'*’ An evaluator 
holds the responsibility of ensuring that 
trust is maintained; thus, he or she must 
consider the risks that evaluation tasks may 
impose on work, immigration, or respect to 
the individual, family, or culture. 

At times, the process to ensure trust 
among program recipients may interfere 
with the evaluation timeline or resources 
available for the evaluation. Nonetheless, it 
is an essential evaluation component to be 
undertaken. If trust is neglected, the fate or 
validity of the evaluation and participation 
in future evaluations by this population 
may be compromised. For instance, one of 
the end products of program evaluation is 
a written report. Forgoing culturally sensi- 
tive reporting of findings increases the risk 
of stigmatization of participants and their 
community. This lack of sensitivity not only 
harms the participants, but also discredits 
gatekeepers diminishing future opportuni- 
ties for evaluation. 

Informed Consent, Language 
and Literacy 

Another ethical issue is that the informed 
consent process, including documentation 


of consent, may not be well understood 
in a population with low literacy or with 
language barriers. The informed consent 
process is a means to inform program recipi- 
ents of the purpose of the evaluation with 
some discussion of the evaluation process, 
risk, benefits and options. “Fn/ormed consent 
is the process of communication between the 
researcher and the participant in which all 
information that could affect willingness to 
participate is made clear and agreement to 
participate is obtained. Written consent is the 
evidence that this communication process has 
been accomplished.”^’' Some authorities' ar- 
gue whether “informed consent” is possible 
given its current structure in a population 
where limited English proficiency and low 
literacy are prevalent. Today’s consent forms 
have morphed into complex and multiple- 
page documents containing medical and 
legal jargon that McDermott and Sarvela'**P- 
‘'®* state “might challenge even the most 
educated and literate of patients.” Despite 
translation to Spanish or having an inter- 
preter present, the content of the consent 
form, or the informed consent process, 
may still be misunderstood due to regional 
differences in the Spanish language (i.e., 
meaning variations of a particular word), 
cultural differences, or Spanish illiteracy.' 
Thus, evaluators must undertake all mea- 
sures (e.g., pilot testing,'® Brislin’s seven-step 
translation process,'’ or use of health literacy 
tools from nationally accredited institutes) 
to ensure individuals understand the evalu- 
ation, including their options and measures 
to protect them from harm. Understanding 
the population’s value and belief systems will 
further support these efforts, increasing the 
possibility of an informed decision. 

Another ethical issue in the informed 
consent process is protection of participants’ 
confidentiality. Most research institutions, 
hospitals, governmental agencies and other 
institutions mandate a signed consent form. 
At times, individuals are asked to provide 
identifiers such as phone numbers or social 
security numbers. This process may increase 
the participants’ risk of being exposed by 
chance or as mandated by law. Furthermore, 
a signature may be interpreted as a binding 


American Journal of Health Education — July/August 2010, Volume 41, No. 4 2S3 



s 

commitment to the study, which may lead 
to low accrual or anxiety among program 
recipients.' Moreover, there may be an in- 
herent pressure to provide this information 
in fear of being identified as a person with 
questionable documentation to reside in 
the country; an ethical issue largely unique 
to Hispanics residing in rural U.S. settings. 
When working with a vulnerable popula- 
tion, evaluators may choose to waive the 
informed consent to protect individuals’ 
identities. However, this choice can be risky 
as evaluators have no documentation that 
risks, benefits and options were explained 
in case adverse events occur. 

Other Culture Issues 

According to Newman and Brown:“ 

149) “Participants in more authoritarian cul- 
tures may be accustomed to acquiescing to 
requests for information coming from sup- 
posed official representatives. They may fear 
recriminations if they do not participate [in 
research or program evaluation] .” Cooper et 
al.' noted this observation in their review of 
studies involving migrant farmworkers. In 
one study, a 100% participation rate was ob- 
tained. Further investigation demonstrated 
that the topic under investigation pertained 
to the participants’ working environment. 
Consequently, participants felt an inherent 
pressure to be in the study because of a real 
or perceived threat to their job security. 
Similar reactions may also occur in differ- 
ent settings when a person of higher social 
class or an authority figure (e.g., physician, 
researcher wearing a white coat or smock, 
or another authoritative figure) requests 
participation in a study. 

Another prevalent issue pertains to ac- 
cess to health care and low-income status. 
Rural settings are known to have limited 
resources for health care’ and lack many of 
the most-needed services, such as low-cost 
health care or psychosocial support in their 
native language.'"’^'’’’ Hispanics in this setting 
are at significant disadvantage due to the 
lack of culturally-competent care, their low 
awareness of and access to services, or lack of 
insurance because of improper documenta- 
tion status or limited job opportunities in 
settings that provide health insurance.’’'’ In 
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the face of limited resources, a solicitation to 
participate in a study may be misunderstood 
as a valuable opportunity to receive medical 
care.' Participation also may be driven by 
program recipients’ economic state such 
as remuneration to meet economic needs. 
For instance, if monetary remuneration is 
purposely or unintentionally set high, it 
may unwillingly compel a low- income per- 
son to participate to meet a financial need. 
Consideration of these ongoing issues must 
be acknowledged in the program evaluation 
plan to ensure program participants are not 
coerced unwittingly. 

Migrant Nature of Hispanics 
in Rural Settings 

Although Kandel and Cromartie’ report 
an increased settlement in rural areas of the 
country, the high mobility of the popula- 
tion, especially among farmworkers, is still 
prevalent.'’’" Applying a follow-up compo- 
nent to evaluation becomes challenging in 
a population with no permanent address or 
contact information. Furthermore, in cer- 
tain situations, collecting contact informa- 
tion is not always feasible or recommended, 
as previously noted. Thus, evaluators may 
have to choose a less sophisticated evaluation 
design to ensure completion of data as set 
by stakeholders.’ 

Advocate versus Evaluator 

Another ethical issue encountered by 
evaluators is role conflict: evaluator versus 
advocate. For instance, programs offered to 
Hispanics in rural settings often are funded 
only temporarily or with other limitations. 
A stakeholder may request that an evalua- 
tion take place to encourage further fund- 
ing.’ However, evaluation findings may not 
support the stakeholder’s intent.’ Thus, 
the evaluator faces an ethical dilemma in 
choosing the evaluator role (e.g., reporting 
actual findings) versus the advocate role 
(e.g., promoting the strengths of a program 
and the needs being served). This decision 
is particularly important in settings where 
limited options are available. As an illustra- 
tion, if a number of people do not have 
the economic or other resource means to 
obtain follow-up care upon the discovery 
of possible disease or injury (e.g., a suspi- 


cious lump identified during a free mam- 
mography screening), it would be unethical 
to discontinue the only local program that 
provides this follow-up care at no cost to 
low-income individuals” because quotas 
outlined in the program objectives (i.e., 
provide care to 1,000 men and women) are 
not met. Discontinuing programs valued 
by the community may cause more harm 
than good and may discredit future evalua- 
tors. This consequence must be considered 
because beneficence and non-malficence 
are vital and well-established components 
of research ethics. 

Other Ethical Issues in Program 
Evaluation 

Other ethical issues common to program 
evaluations apply in the context of Hispan- 
ics residing in rural settings as well. These 
issues include the question of the external 
evaluator versus the internal evaluator, 
and participants’ reactivity and the effect it 
has on the validity of the evaluation.’ The 
debate of internal versus external evaluator 
revolves around issue bias, validity of results 
and cost (i.e., outside consulting versus us- 
ing a staff member). The use of an internal 
evaluator may be criticized as being biased 
because he or she is highly vested in the 
program’s performance.'* Yet, this approach 
can increase the validity of the data due to a 
deeper understanding of the program and 
population dynamics’ as well as program 
context.'* Furthermore, program recipients 
may be more inclined to participate because 
the important element of trust is likely to 
have been established. Gaining an insider 
perspective of issues that affect programs 
with underserved Hispanics and trust are 
more challenging for an outsider evalua- 
tor. Nonetheless, outsider evaluators may 
identify issues or recommendations from 
an unbiased or dispassionate perspective, 
thereby introducing a perspective that can 
assist in improving program efforts. 

Another ethical issue to consider is re- 
activity, e.g., when a participant responds 
or acts differently because of awareness of 
the researcher or the investigative setting. 
Several common examples of reactivity ex- 
ist - i.e., researcher or observer effect, the 
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Hawthorne Effect, the John Henry Effect, 
social desirability response bias, and others.'* 
If reactivity is not controlled, it can affect the 
validity of the evaluation. A partial solution 
to this concern is to establish trust, as noted 
previously. Participants may view the evalu- 
ator as a threat; and thus, feel intimidated to 
respond and express their views freely. Or, 
as discussed earlier, participants may not 
understand the question due to issues of lit- 
eracy. Cultural differences may also affect the 
validity of response.' Whereas participants 
may think they comprehend what is being 
asked, they are inferring a different mean- 
ing; not that intended by the evaluator. One 
mechanism to resolve this misunderstanding 
is through thorough pilot testing of ques- 
tions with program recipients.'* 

AN EXAMPLE OF A STRATEGY TO 
ADDRESS ETHICAL CHALLENGES 

A program currently operating in the 
Tampa Bay (Florida) Area is Catholic 
Charities’ Catholic Mobile Medical Service 
(CMMS). It serves as a prime illustration 
of the need for sensitivity and cultural 
competence during program evaluation. 
Approximately 50% of CMMS’s popula- 
tion served is comprised of migrants living 
in rural Hillsborough County (the large 
geographically and demographically diverse 
county that includes Tampa). Its patient 
population includes individuals with low 
literacy (Spanish and/or English), mixed 
documentation status, low income, and who 
are economically challenged. A previous 
evaluation conducted by the program staff 
revealed transportation deficits and cost of 
care as the major barriers to health services. 
The director of this program also reports 
that program participants do not seek health 
services, even when illness occurs, because of 
fear of losing their employment. For future 
evaluations with program participants, some 
considerations that should be undertaken to 
reduce participant burden and exploitation 
may include: (1) scheduling data collec- 
tion in the evening or on the weekend at a 
convenient location; (2) discussing with the 
program director the goal for the evaluation 
and suggesting an evaluation that highlights 


their assets and ways to improve their ser- 
vices; (3) sharing all translated data collec- 
tion forms and the informed consents forms 
with program staff to assess comprehension; 

(4) discussing with the program director ac- 
ceptable remuneration that is not coercive; 

(5) finding times to pilot test all data collec- 
tion forms; and (6) considering the use of 
a trusted staff member to assist with data 
collection and participant recruitment. 

CONCLUSIONS 

Evaluations of health-related programs 
can impact their eventual fate and outcomes. 
Most importantly, they can have a direct 
effect on the people the program serves. 
Given that Hispanics living in rural settings 
have certain vulnerabilities for exploita- 
tion; evaluators must strive for the highest 
ethical standards in all phases of program 
measurement and evaluation. Attention to 
the sensitive issues and cultural nuances 
that surround this population will support 
these efforts. 

The discussion presented herein has the 
following implications for evaluation of 
health-related programs: 

• A skilled evaluator with experience and 
training in cultural competence with Hispan- 
ics, especially those living in rural settings, 
is necessary for an ethical and efficacious 
evaluation.^ 

• Employing a skilled individual who is 
well- versed in program evaluation, including 
its technical capabilities and interpersonal and 
communication skills, should be a priority. 

• Evaluators should confirm if remunera- 
tion is justified, especially with low-income 
communities. What an evaluator may think 
is a small or modest payment may actually be 
the compelling factor for participation.' 

• The evaluator shares responsibility for 
gaining and maintaining the trust of the 
community and important gatekeepers. For 
example, in an evaluation of citrus worker 
health issues. Tuque et al.’ noted their disclo- 
sure to farmworkers during the evaluation to 
reassure them that they did not work for the 
citrus companies, and that neither commu- 
nity health workers nor other workers would 


be penalized if they refused to participate in 
the project. 

• Evaluators should always strive to use 
an informed consent process when possible. 
Situations may arise when a signed informed 
consent is not feasible. Moreover, recording a 
person’s name may not be necessary. Finding 
creative ways to collect an informed consent 
form while protecting an individual’s identity 
maybe a viable option, such as using identifiers 
that only the participant would recognize.^ 

• Informed consent forms and all evalua- 
tion instruments should be linguistically and 
culturally relevant. Use of pilot-testing is a 
good means to confirm accuracy of language 
and meaning on forms. Some evaluators'* ar- 
gue for deliberate “cultural taHoring”^^ of ques- 
tions or use of an extension of back-translation 
known as “decentering”“ to assure relevance. 
Other avenues include using health literacy 
tools from nationally accredited institutes to 
improve readability of consent forms. 

• Always strive to reduce participant burden 
- for example, providing evaluations at times 
when participants are at the program loca- 
tion or go to their homes. Evaluators must be 
cognizant of making evaluations convenient 
and allowing the flexibility to work around 
the participant’s schedule. 

• In reporting evaluation findings, consider 
the impact and ethical implication this report- 
ing will have on the participants and programs. 
Even if evaluation findings conclude that a 
program is ineffective in some of its intended 
effects, it may have other values and attributes 
needed by participants. 

Whereas this paper attempted to sum- 
marize the unique ethical issues with His- 
panics residing in rural settings, the review 
was limited to literature mostly involving 
migrants or farmworkers. A comprehensive 
list of ethical guidelines for use by program 
evaluators working with ethnic minori- 
ties, specifically in rural settings, may be 
warranted. There is also a need to develop 
strategies to ensure that program recipi- 
ents are well informed and understand the 
informed consent process and evaluation 
procedures. Finally, the sensitive nature of 
this population highlights the importance 
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of cultural competence training for evalu- 
ation staff 
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